Zero-shot keyword spotting for visual speech recognition in-the-wild

Visual keyword spotting (KWS) is the problem of estimating whether a text query occurs in a given recording using only video information. This paper focuses on visual KWS for words unseen during training, a real-world, practical setting which so far has received no attention by the community. To this end, we devise an end-to-end architecture comprising (a) a state-of-the-art visual feature extractor based on spatiotemporal Residual Networks, (b) a grapheme-to-phoneme model based on sequence-to-sequence neural networks, and (c) a stack of recurrent neural networks which learn how to correlate visual features with the keyword representation. Different to prior works on KWS, which try to learn word representations merely from sequences of graphemes (i.e. letters), we propose the use of a grapheme-to-phoneme encoder-decoder model which learns how to map words to their pronunciation. We demonstrate that our system obtains very promising visual-only KWS results on the challenging LRS2 database, for keywords unseen during training. We also show that our system outperforms a baseline which addresses KWS via automatic speech recognition (ASR), while it drastically improves over other recently proposed ASR-free KWS methods.Comment: Accepted at ECCV-201

Maximum mutual information design for amplify-and-forward multi-hop MIMO relaying systems under channel uncertainties

Conference Theme: PHY and FundamentalsIn this paper, we investigate maximum mutual information design for multi-hop amplify-and-forward (AF) multiple-input multiple-out (MIMO) relaying systems with imperfect channel state information, i.e., Gaussian distributed channel estimation errors. The robust design is formulated as a matrix-variate optimization problem. Exploiting the elegant properties of Majorization theory and matrix-variate functions, the optimal structures of the forwarding matrices at the relays and precoding matrix at the source are derived. Based on the derived structures, a water-filling solution is proposed to solve the remaining unknown variables. © 2012 IEEE.published_or_final_versionThe 2012 IEEE Wireless Communications and Networking Conference (WCNC), Paris, France, 1-4 April 2012. In IEEE Wireless Communications and Networking Conference Proceedings, 2012, p. 781-78

Joint robust weighted LMMSE transceiver design for dual-hop AF multiple-antenna relay systems

In this paper, joint transceiver design for dual-hop amplify-and-forward (AF) MIMO relay systems with Gaussian distributed channel estimation errors in both two hops is investigated. Due to the fact that various linear transceiver designs can be transformed to a weighted linear minimum mean-square-error (LMMSE) transceiver design with specific weighting matrices, weighted mean square error (MSE) is chosen as the performance metric. Precoder matrix at source, forwarding matrix at relay and equalizer matrix at destination are jointly designed with channel estimation errors taken care of by Bayesian philosophy. Several existing algorithms are found to be special cases of the proposed solution. The performance advantage of the proposed robust design is demonstrated by the simulation results. © 2011 IEEE.published_or_final_versionThe 2011 IEEE Global Telecommunications Conference (GLOBECOM 2011), Beijing, China, 5-9 December 2011. In Globecom. IEEE Conference and Exhibition, 2011, p. 1-

Vibration analysis of a beam on a moving vehicle under the road excitation with different contact models

Dynamic analysis of a beam on a moving vehicle is presented in this paper. The vehicle is simulated by a four degrees-of-freedom mass-spring system and the beam on top is supported by spring-damping systems. Two contact models named the ‘point contact’ and the ‘patch contact’ respectively, are adopted to simulate the interaction between road surface and vehicular tyres. The equation of motion of the beam-vehicle system is formulated and the dynamic response on the beam under the excitation of the irregular road surface is derived. Numerical simulations are conducted to demonstrate the influence of different factors, such as the length of the contact, the velocity of vehicle, the road condition and the bracing stiffness, etc. on the vibration level of the beam structure, which aims to provide references on the vibration problem in transporting a beam-shaped package